Semantic Video Transcoding Using Classes of Relevance

In this work we present a framework for on-the-fly video transcoding that exploits computer vision-based techniques to adapt the Web access to the user requirements. The proposed transcoding approach aims at coping both with user bandwidth and resources capabilities, and with user interests in the video’s content. We propose an object-based semantic transcoding that, according to user-defined classes of relevance, applies different transcoding techniques to the objects segmented in a scene. Object extraction is provided by on-the-fly video processing, without manual annotation. Multiple transcoding policies are reviewed and a performance evaluation metric based on the Weighted Mean Square Error (and corresponding PSNR), that takes into account the perceptual user requirements by means of classes of relevance, is defined. Results are analyzed varying transcoding techniques, bandwidth requirements and video types (with indoor and outdoor scenes), showing that the use of semantic can dramatically improve the bandwidth to distortion ratio.